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ABSTRACT 


Robotics is a research topic among the most funded around the world and robots have the potential to 
largely reduce human labors. The design of robots is, on the other hand mental labor intensive. It is 
surprising that automatic robot design, as an important research topic, remains unexplored till today. 
In this paper, a automatic design framework is proposed, where the robotics design problem is model 
as a black-box function optimization process. Given robots parts and a specific purpose that the robot 
is designed to accomplish, the framework tries to auto-assemble the parts in a way that is best for the 
purpose. The framework is a simulation and optimization interactive process, where a physics engine 
or even real world simulations are employed to score a assembling setting according to its capability 
to accomplished the given task, and a black-box optimization method is employed to optimize the 
score. Bayesian hyper-parameter optimization is used to carry out the optimizations, for its advantage 
of requiring less sampling than other methods. Experiments show that the proposed method is able to 
find acceptable designs of a two joint system within only 5 simulations. 
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1 Introduction 


It is always desirable for Artificial intelligence (AJ) to carry out more work for humans. AI, as its physical counter part 
such as automated machines, has the potential to largely reduce human labors. It is desirable that AI can automatically 
create arts for entertainments and industrial purposes. Robotics is a combination of mechanics and computer science, 
and is one of the most funded research areas in AI around the world. Despite the fact that robot control is a well 
established area, the automatic robot design problem receives much less attention. In this work, a simulation based 
automatic design method is proposed. The results can be evaluated in real world situation with 3d printers. 


In past works, there are studies that consider the automatic design of controlling logic, i.e. the “software” part of robots 
{1]). In [P], auto assembling problem is studied, however, this study is for constructing industry where almost all contact 
of parts are fixed in position, thus it is a different problem from robot construction, where most parts are not fixed. 


There are two major problems in automatic robot designs, data representation and optimization efficiency. It is well 
known that robots can be represented as a tree structure (as in ROS [3]) of individual parts. The tree structure is a 
discrete data structure and parts have continuous properties. The fusion of discrete and continuous data representations 
can be performed for the construction of a continuous search space. Because of the fact that evaluations of robot designs 
are expensive, the search algorithm in the constructed search space should be able to reach optima with a few steps of 
evaluations. 


In this paper, the automatic design of robotics is studied. Instead of designing controlling software, or hardware 
structures for a specific purpose, the auto assembling procedures for general purpose robots are devised. 
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2 Methodology 


Designs of physical structures in real world have a long history of billions of years. One reason is that the evaluations 
of the fitness of one design (in this case, a physical structure of a species) is slow. The evolution speed of physical 
structures is partly dependent on evaluation speed. One evidence is the fruit flies widely used in animal experiments. 
Due to their high reproduction speed, evolutions which usually take generations can be accomplished in a short time. 


In the computer world, the physical structure evolution can be almost infinitely fast because the seemingly unlimited 
computing power that humanity can command. In this work, the robot design is helped with accelerated physical 
simulations. The framework is illustrated in Fig.|1| The optimization process generates a vector, which is then translated 
into a physical robot representation. Then this representation is trained with reinforcement learning according to a 
specified task. After that, the model is evaluated so that how well it can accomplish the task. And the evaluation score 
is optimized. The optimizer should have the ability to reach optimal with minimum evaluations, which is similar to 
active sampling to minimize sample counts. Thus, the framework is a active design procedure. 
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Figure 1: Active design: automatic robot design with simulation and optimization interaction 


The input of active design is a set of parts and their specifications S, a reinforcement learning method £, an evaluation 
function Score(S, J, £) that determines how well the robot can accomplish a specified task while the parts are connected 
with joints J, and an optimizer 


argmax Score(S, L) (1) 
S 


Due to the challenges of representation and efficiency as described in the introduction section, two problems need to 
be addressed in this framework. First, how to change the vector representation, which is the operating objectives of 
black-box optimizers, to the tree structure recognized by physical simulators or real robot constructors. Second, which 
optimizer to choose to minimize the evaluations of the Score function. 


2.1 Representation 


The representation need to be two ways. One is from tree structured data to vector representation, the other is from 
vector to tree structured parts and their specifications. First, let us see the first transformation, which is to combing a 
similarities of specification matrix C and a similarity matrix G defined by the tree structure. If structure a is different 
from structure b, despite their similarities of parts specifications, their distance in the similarity space will be larger than 
c, which has the same tree structure as a. 


The fusion of two similarity matrices should preserve C, which is Euclidean distances as well as possible, while 
following the constraints in G = {g;; } calculated by graph similarities. 


Then the problem of fusing two matrices is formalized as 
‘ 4 2 
PEE- e) 
tag 


8.t. VJij, gmn € Gif Jij S Imn, Gy 2 Can 


(2) 


Because that similarity matrices are symmetric, the upper half of G is extracted and put in a vector M = {mx} and sort 


in decreasing order; and add another two vectors D = {dx}, K = {k,} for each element mz, = gij, dk = Pij, kk = Cj 
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Then, the optimization task of Eq. (2) is transform as 


* o i 2 
magne) di) 


st.0 < ky < koe < kje 


(3) 


which is equivalent to 


1 
min -KTK — DTK 
K 2 (4) 
8.t.0 < ky < koee kje 


This is a quadratic programming process. The reverse transformation can be accomplished in two steps. First, find the 
best structure, then find the specifications. Structure can be found by nearest neighbor in the transformed space, since 
structure similarity are fully preserved. After that, there is no difficulty in finding the specifications. 


2.2 Optimization 


Bayesian optimization [5] is chosen to perform the main work of active design, because of its ability to reach optima in 
a few search steps. This optimization model works under the assumption that parameters to be selected and the score 
to be maximized form a Gaussian process, i.e. that they form a Gaussian distribution. The process of optimization 
combined with the simulation steps are given in Algorithm. 


Algorithm 1 Active design of robots with Bayesian optimization 


Input: Specifications {S}, a reinforcement learning method £, an evaluation function S'core(S, J, £), maximum 
evaluations N. 
Output: Robot structure and joint specifications. 
1: Set GP priors. 
2: Generate two space filling J and structure settings, calculate their vector representations with procedures in Section 
and initialize GP with generated vectors and their scores. 
3: Add the calculated vectors vı € V, v € V and their scores to a observation set as O = 
{vT , Score(v1)]*, [vd , Score(v2)]7 }. 
4: while n < N do 
5: Update GP posterior with observed vector representations and their scores in O. 
6: From GP, calculate a position v’ € V in the space of robot vector representations that maximizes the GP 
posteriori. 
7: Evaluate Score(v’), and add it to O as [vf , Score(un)]|". 
8: end while 


9: Output as argmax  Score(vn) 
[vT ,Score(un)]7 €O 


3 Experiments 


Two toy problems are studied with the ROS and pybullet{6] simulation tools. In the first problem, the input is tree links 
with the cylinder shape, and joint positions at the end. The task is to “stand up”, which means the system will try to 
lift one of the pieces up, while other two pieces touch the ground. Random initialization with a simple reinforcement 
learning set up would result in a situation in Fig. After 5 evaluations with active design, the robot have achieved 
the gesture that gives highest score, i.e. that the central link is highest than other gestures as illustrated in Fig. The 
reinforcement method of deterministic policy gradient |/7] is selected in experiments. In another problem, the system try 
to form a gesture of diagonal. Random setup would result in Fig. because rotation axises of joints do not allow a 
diagonal gesture. After 7 steps of active design. The gesture is accomplished as in Fig. 


Comparisons are carried out with the scipy [8] optimization method Nelder-Mead [9]. The scipy module can achieve 
similar results to Bayesian optimization, but with more sampling. The results are given in Table[I] 
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(a) random set up (b) joint optimized with active design 


Figure 2: Problem one, optimize joint configurations so that the central link can stay as high as it can be. 


(a) random set up (b) joint optimized with active design 


Figure 3: Problem two, optimize joint configurations so that a diagonal gesture can be performed. 


Table 1: Function evaluation times comparison between Bayesian optimization and scipy’s optimization module. 


simulations run 


Optimization method problem one problem two 


proposed 5 7 
scipy 112 128 


Each simulation takes about 10 seconds on a mainstream PC. As a result, the proposed method can finish design in 
minutes, while hours are required using the scipy module. This is because many optimization methods are designed 
under the assumption that function evaluations are fast, which is not the case in robot simulations or evaluations. 
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